SUPPORT PROPERTIES AND HOLMGREN'S UNIQUENESS THEOREM 
FOR DIFFERENTIAL OPERATORS WITH HYPERPLANE 

SINGULARITIES 



GESTUR OLAFSSON AND ANGELA PASQUALE 

Abstract. Let W be a finite Coxeter group acting linearly on R n . In this article we 
study support properties of VF-invariant partial differential operator D on K n with real 
analytic coefficients. Our assumption is that the principal symbol of D has a special form, 
related to the root system corresponding to W. In particular the zeros of the principal 
symbol are supposed to be located on hyperplanes fixed by reflections in W. We show that 
conv(supp Df) — conv(supp /) holds for all compactly supported smooth functions / so 
that conv(supp/) is VT-invariant. The main tools in the proof are Holmgren's uniqueness 
theorem and some elementary convex geometry. Several examples and applications linked 
to the theory of special functions associated with root systems are presented. 



Introduction 

Let D be a linear partial differential operator on MJ 1 with constant coefficients. Then a 
classical theorem of Lions and Titchmarsch states that, for every distribution u on R n with 
compact support, the convex hulls of the supports of Du and u are equal: 

conv(supp -Dm) = conv(suppw), u G E'iW 1 ). (1) 

By regularization, this is equivalent to stating that for every compactly supported smooth 
function / on M. n , the convex hulls of the supports of Df and / are equal: 

conv(supp£>/) = conv(supp /), / G Cf (R n ). (2) 

In fact, Lions |Lio51j proved a more general version of the support theorem, namely 

conv (supp(f * u)) = conv(suppt>) + conv(suppu), Wv, u G £'(W l ). (3) 

We refer to [Boe85j for an elementary proof of this theorem. The first version obviously 
follows from the third one © by taking u = So, the delta distribution at the origin. 

The comparison of the supports of / and Df plays an important role at several places 
in analysis. One typical situation is the study of solvability of differential operators. Recall 
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that a linear partial differential operator 

D= a i( x ) dI ( 4 ) 

\I\<m 

with smooth coefficients aj is said to be solvable in R n , provided .DC°°(R n ) = C°°(R n ), that 
is, if for every g G C°°(R n ) the differential equation Df = g has a solution / G C°°(R n ). 
The following theorem gives a necessary and sufficient condition for the solvability of D (see 
|Tre75j . Theorem 3.3). 

Theorem 0.1. Let D be a linear partial differential operator with smooth coefficients in R n . 
Then D is solvable if and only if the following two condition are satisfied: 

(a) (semi-global solvability) For every compact subset K ofW 1 and for every g G C°°(R") 
there is a function f G C°°(R n ) so that Df = g on K . 

(b) (.D-convexity of W 1 ) For every compact subset K ofW 1 there is a compact set K' so 
that for every f G C£°(R") the inclusion supp(_D*/) C K implies supp / C K' '. Here 
D 1 denotes the formal transpose of D. 

The support theorem of Lions and Titchmarsch implies that, for every linear partial dif- 
ferential operator D with constant coefficients, R™ is D-convex, and that we can in fact take 
K' = conv K. Observe that, in this case, condition (b) also implies that the operator D l is 
injective on C°°(R n ). 

The main result of this paper is Theorem 12.11 which provides an extension of the support 
theorem of Lions and Titchmarsch to specific (but very large) classes of invariant singular 
linear partial differential operators with real analytic coefficients and to distributions having a 
compact support with invariant convex hull. The invariance considered here is with respect 
to certain finite groups of orthogonal transformations generated by reflections. In the 2- 
dimensional case, examples of such groups are the groups of symmetries of regular n-agons. 
The principal symbols of the examined differential operators are allowed to vanish, but only 
in a precise way, along the reflecting hyperplanes. See formula (jl(J|) below. 

Some restrictions in generalizing the theorem of Titchmarsch and Lions are of course 
needed. The following easy example shows that the theorem cannot hold for arbitrary linear 
differential operators with variable coefficients, even in the one-dimensional case and with 
real analytic coefficients. 

Example 0.2. Consider the differential operator D = x-^ on R. Let u = X[o,i] De the 
characteristic function of the interval [0, 1] . Then Du = —5i, where 5\ denotes the delta 
measure at 1. Therefore supp(-D-u) = {1} and suppn = [0, 1] are convex and different. 

In the one-dimensional case, the differential operator D = x-^ of Example 10.21 belongs to 
the class of differential operators to which our support theorem applies. Observe that D is 
an even differential operator which is singular at x = 0. For this very special differential 
operator, our theorem states that conv (supp Dm) = conv(suppw) if u G £'(R) satisfies one 
of the following conditions: 

(1) suppw c]0, +oo[ , 

(2) suppw c] — oo,0[, 

(3) conv(supp-u) is symmetric with respect to the origin 0. 

Of course the distribution X[o,i] from Example ID. 21 does not fulfill any of these conditions. 
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The core of the proof of Theorem 12. II is to show that the considered situation allows us to 
apply Holmgren's uniqueness theorem for comparing the size of the supports of / and Df 
when / is a compactly supported smooth function with the property that conv(supp/) is 
invariant. The employ of Holmgren's theorem is the reason for imposing to the coefficients 
of the considered differential operators to be real analytic. 

Several other authors employed Holmgren's uniqueness theorem for proving D-convexity 
properties (see e.g. jRW76| . |Cha79j . |vdBS93j. Remarkable is nevertheless, that the proof 
of our theorem is very elementary. It requires only basic facts from convex geometry and an 
application of equation (J3J). 

The article consists of two parts. The first part contains the proof of Theorem 12.11 which 
does not require any knowledge of symmetric spaces. In the second part, we discuss several 
applications of our main results, including hypergeometric differential operators, Bessel dif- 
ferential operators, shift operators, Hamiltonian systems, and invariant differential operators 
on symmetric spaces. 

The solvability of G-invariant differential operators is one of the fundamental problems 
in the analysis on a symmetric space G/H (see [Hel84 , p. 275). Recall that Theorem ID. II 
holds more generally if W 1 is replaced with a 2nd countable smooth manifold (see |Tre75j , 
p. 14). Since G/H is a second countable smooth manifold, one obtains the equivalence 
between global and semi-global solvability, provided one can prove that G/H is D-convex. 
The solvability of invariant differential operators on Riemannian symmetric space was proved 
by Helgason |He73j . In the general pseudo- Riemannian symmetric case, van den Ban and 
Schlichtkrull [vdBS93] determined a sufficient condition for a G-invariant differential operator 
D ensuring that G/H is D-coiwex. This condition involves the degree of the polynomial 
which is the image of D under the Harish-Chandra isomorphism. It is for instance always 
satisfied when G/H is split, i.e., has a vectorial Cartan subspace. The Riemannian symmetric 
spaces of the noncompact type are examples of split symmetric spaces and so are the K e 
space of Oshima and Sekiguchi, [OS80 . 



As a first application of Theorem 12.11 we deduce in Section 0] the D-convexity of Rie- 
mannian symmetric spaces G/K of noncompact type when D is a G-invariant differential 
operator on G/K. Our method, which is based on taking the radial component of D along 
the Cartan subgroup, is different from those of used in |He73j and [vdBS93 . 

Support theorems play also an important role in harmonic analysis, in particular in Paley- 
Wiener type theorems. These theorems characterize the space of functions which are im- 
age, under a suitable generalization of the Fourier transform, of the compactly supported 
smooth functions. Applications in this setting appeared first in the work of van den Ban and 
Schlichtkrull on Fourier transforms on pseudo- Riemannian symmetric spaces |vdBS97j . The 
basic idea, which we shall outline more precisely in Section @J is to cancel the singularities 
appearing in a wave packet / by applying a suitable differential operator D. The problem is 
to compare the size of the support of Df, which can be easily determined, with the -hard to 
determine- size of the support of the original wave packet /. In fact, our need for support 
properties like those stated in the present paper turned up in the proof of a Paley- Wiener 
type theorem for the B-hypergeometric transform, which is a Fourier type tra nsform related 



to the theory of hypergeometric functions associated with root systems. See |OP04| . How 



ever, we point out that Theorem 12 .11 is stated in a very general setting and applies to many 



3 



different situations. See Example 11.31 and the Subsections 13 . 11 l3~2l and l3~3l for several special 
cases. 

Our paper is organized as follows. In Section ^ we introduce the general setting in which 
our extension of the theorem of Lions and Titchmarsch will be proved. The main results, 
Theorems 12.11 and 12.21 wm be stated and proved in Section |21 Section |H] presents several 
concrete situations where our support theorem applies. The presented examples are related 
to the theory of special functions associated with root systems. The last section is devoted 
to applications. We deduce the Z)-convexity of Riemannian symmetric spaces of noncompact 
type when D is an invariant differential operator. Moreover, we describe how to employ The- 
orem 12.21 for proving of Paley- Wiener type theorems in the harmonic analysis on symmetric 
spaces and on root systems. 

I. Notation and setting 

1.1. Finite Coxeter Groups. In this section we introduce the notation and set up that 
will be used in this article. In particular we introduce the class of differential operators that 
will be considered in this article and give few examples. 

In the following a stands for an real Euclidean vector space of dimension n, i.e., a ~ R n . 
Furthermore D will stand for a differential operator on a with analytic coefficients. We 
denote by (•, •) a (positive definite) inner product on a. Set \x\ = ■>/ (x, x). For e > we 
denote by B e := {x G a : \x\ < e} the closed Euclidean ball in a with center and radius e. 

Let a* denote the real dual of a. For each a G a* \ {0} we denote by y a the unique element 
of a satisfying a(x) = (x, y a ) for all x G a. We set 

x a := 2y a /(y a ,y a ) (5) 

and notice that x a is independent of (•, •). With each a G o* \ {0} we associate the reflection 
r a of o across the hyperplane TC a : = kera. Thus 

r a( x ) — x ~ a(x)x a , x G a. (6) 

A finite set A C a* \ {0} is called a (reduced) root system if the following conditions holds 
for A: 

(Rl) If a G A, then A n Ra = {±a}; 

(R2) If a, p G A, then r a (j3) G A. 
The elements of A are called roots. Observe that we are not requiring that A contains a 
basis of o*. In particular, our definition allows A to be the empty set. 

A subset II of A is called a simple system if II is linearly independent and if any root in 
A can be written as a linear combination of elements in II in which all non-zero coefficients 
are either all positive or all negative. If A = 0, then we set II = 0. 

Let a simple system II of A be fixed. Set A+ := E> II n A, where R + = {x G R : x > 0}. 
The elements in A + are said to be positive. Observe that —a = r a (a) G A for all a G A. 
Hence A = A+U(-A+). 

Let W C GL(o) be the group of orthogonal transformations of a generated by the re- 
flections {r a : a G A}. It coincides with the group generated by {r a : a G II}. We set 
W = {id}, if A is empty. If dim a = 1 and A ^ 0, then W = {±id}. The group IF is a 
finite Coxeter group. Conversely, every finite Coxeter group originates from a root system 
as above. Sec e.g. [GP00J, Chapter 1. Among the finite Coxeter groups we find the Weyl 
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groups (for instance the finite groups of permutations) and the dihedral groups (that is, the 
groups of symmetries of the regular n-gons). 

The group W acts on functions / : a — > C according to 

(w ■ f)(x) := f{w~ 1 x), weW,xea. (7) 

It also acts on compactly supported distributions u G S'(a) and on differential operators D 
on a by: 

<{wu),f> :=< u,w- 1 -f>, weW,fe C°°(a) , (8) 
(w ■ D)f := w ■ D(w~ l • /), weW,fe C°» , (9) 

where we have written the pairing between distributions and functions by < u, f >:= u(f). 
The function / (resp. the compactly supported distribution u or the differential operator 
D) is said to be W -invariant provided w ■ f = f for all w G W (resp. w - u = u or w ■ D = D 
for all w G W). For instance, if dim a = 1 and A ^ 0, then P^-invariant means even. More 
generally, let \ be a character of W. Then / is said to transform under W according to 
X if w • f — x( w )f f° r a ^ w ^ W. This definition extends similarly to distributions and 
differential operators. Notice that if / transforms under W according to a character x, then 
the support of / is ^-invariant. 

1.2. X-elliptic polynomials and X-elliptic differential operators. We will be studying 
differential operators with leading symbol of a specific form. We will therefore need the 
following definition: 

Definition 1.1. Let X be a non-empty P^-invariant convex open subset of a, and P : 
a x a* — ► C a polynomial function. We say that P is a homogeneous X -elliptic polynomial 
if the following holds: 

(PI) P is a homogeneous polynomial in A G o* with real analytic coefficients on a, i.e. 
there is m G No so that 

P(x,X) = ^ a/(^)A J 

\I\=m 

where ai(x) is real analytic, and |/| := Y^k=i^ki if I — (h, ■■■ , *n) G Nq is a multi- 
index. 

(P2) If (x, A) G X x a* and A ^ 0, then P(x, A) ^ 0. 

Let D be a W-invariant linear partial differential operator on a with real analytic coefficients. 
We say that D is X -elliptic if its principal symbol is of the form 

a(D)(x,X) = (p(\) Y[[a{x)] n{a) J P(x,X), (x,A)Goxa*, (10) 

where p(X) is a homogeneous polynomial, n(a) G No := {0,1,2,...} for all a G A, and 
P(x, A) is a homogeneous X-elliptic polynomial. 

In this paper we consider X-elliptic linear partial differential operators as operators acting 
on functions or distributions on X. Note that the decomposition of a(D)(x, X) in (jTUjl is 
in general not unique: if a(x) ^ for all x G X, then we can replace P by a(x) k P(x, A), 
< k < n(a), and replace n(a) by n(a) — k. 
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The class of X-ellipltic contains all elliptic partial differential operators on a by taking 
X = a and A = 0. But, more generally, the principal symbols of the considered operators 
are allowed to vanish at the zeros of p(X) as well as along the hyperplanes 7i a with a G A. 
Observe that the condition in (jlUJl imposes a restriction only on the principal part of the 
differential operators. In particular, suppose Di, D 2 are W- invariant linear partial differential 
operators with real analytic coefficients so that deg-Di > deg_D 2 - If D\ satisfies condition 
fjlOp . then the same is true for Di + D 2 . 

Example 1.2. Any partial differential operator with constant coefficients p(D) satisfies (fTU|) 
when we choose X = a and A = 0. Indeed, in this case we do not impose any symmetry 
condition, and a(p(D))(X) = p(X) is of the form (|10p. 

Example 1.3 (The one-dimensional case). Suppose dim a = 1 and A ^ 0. We shall identify 
a = a* with R. The possible subsets X are the open intervals of the form ] — R,R{ with 
< R < +oo. Then the differential operators considered are the even ordinary differential 
operators with real analytic coefficients and with principal symbol 

a(D)(x,X) = x n p(\)P(x,\). 

Here, neNo, the polynomial p(X) is homogeneous, and P(x, A) is homogeneous X-elliptic. 
For X = M, examples are x-^ and the Euler operator 

rn 2 d 2 d . 

E = x -— + a x— + 6, 

where a, b G C. Other examples, which are relevant in harmonic analysis, are constructed 
from the Jacobi and the Bessel differential operators on ]0, +oo[. Let L and L respectively 
denote the Jacobi and the Bessel differential operators, i.e. 



2 



dx 2 



acoth(a;) + 6coth(2x) 



1 d 
dx 



2 



d[_ 

dx 2 



' 1 t 1 
a — h o — 

. x 2x 



d_ 

dx 



Then, for specific positive integral values of the constants a and b, the operator L agrees 
with the radial part of the Laplace-Beltrami operator of a rank-one Riemannian symmetric 
spaces G/K of noncompact type, whereas Lq is the radial part of the Laplace-Beltrami 
operator of the corresponding rank-one Riemannian symmetric space Gq/K of Euclidean 
type. Examples of such spaces G/K are the (real, complex or quaternionic) hyperboloids; 
examples of spaces Gq/K are given by M n regarded as homogeneous space M(n)/0(n). We 
refer to |Hel84j . Ch. II, Propositions 3.9 and 3.13, for more information on radial parts of 
the Laplace-Beltrami operators. 

The operators D := sinh 2 x • L and D := x 2 ■ L satisfy our assumptions since they 
are even operators on R with real analytic coefficients, and a(D)(x, A) = x 2 A 2 ( ijlI ^ ia: ) and 
a(D )(x, A) = x 2 A 2 . Generalizations in more variables of these examples will be treated in 
SectionEl Note that, in the first case, we can take P(x, A) = (^) 2 \ 2 or P(x, A) = (^) 2 . 

Example 1.4 (The Calogero model). The Calogero model is a non-relativistic quantum 
mechanical system of n + 1 identical particles on a line interacting pairwise. Such a system 
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is described by the Hamiltonian 

1 71+1 



1 




2 : 



(pi, . . . ,p„+i), (xi, . . . ,X n+ i) 



n+1 



j=l l<i<j<n+l 



where the positive constant g 2 is the coupling coefficient. The associated Schrodinger oper- 
ator is 



See e.g. |Pe9 0j. (3.1.1) and (3.1.14), I. See also |Ca71j . Let {e±, . . . ,e n+ i} be the standard 
basis of R n+1 and let (•, •) be the standard inner product on R n . For 1 < i, j < n + 1 with 
i 7^ j set «ij := — tj. Then A := {ctij : 1 < i,j < n + 1, i ^ j} is a root system of type 
A n . We take A + := {ctij : 1 < i < j < n + 1} as a set of positive roots. The corresponding 
system of simple roots is II = {<x, J+ i : j = 1, . . . , n}. The finite Coxeter group W associated 
to A n is the group & n of permutations of the set {ei, . . . , e n+ i}. We can now write 



where L a is the Euclidean Laplace operator on o. It follows that 5*cai is W^-invariant. If 
tc(x) := rLeA^' 2 -)' t nen the differential operator 7r(x)S'cai(^) satisfies our requirements in 
(ll(J|) . This example will be generalized in Sectional 

Remark 1.5. Multiplication by sinh 2 x and x 2 in Example 11.31 fresp. by n in Example 11.4)1 . 
which is used to obtain VF-invariant differential operators with analytic coefficients in a, is 
in fact inessential when dealing with even (resp. W- invariant functions). Indeed, despite the 
singularity of their coefficients, the operators L and L map even smooth functions on K in 
smooth functions. Likewise, the operator Seal maps smooth IV-invariant functions on a into 
smooth functions. This remark will be stated, in a more general form, in Section El 

1.3. Function spaces. We now introduce the class of functions that will be considered in 
this paper. As above, let X be a fixed Vr-invariant open convex subset of a. Let S'(X; W) 
denote the space consisting of distributions u on a so that conv(suppw) is a VT-invariant 
compact subset of X, and let C^°(X; W) be the subspace of £'(X; W) consisting of C°° 
functions. Important subspaces of C%°(X;W) are the spaces C^°(X;x), formed by the 
smooth compactly supported functions / : X — > C which transform under W according to 
the character \ °f W . For instance, if \ is the trivial character, then we obtain the space 
C£°(X) W of ly-invariant smooth functions on X with compact support. Similarly, inside 
£'(X] W) we find the spaces £'{X\ \) °f compactly supported distributions on X which 
transform according to \i an d the space £'(X) W of PF-invariant distributions on W with 
compact support. 

Example 1.6. If dim a = 1 and X = a, then we can identify X = a = a* with R. Suppose 
first A = 0. Then W is trivial, and C~(a; W) (resp. £'(a; W)) reduces to the space of C°° 
functions (resp. distributions) on R with compact support. If A ^ 0, then W = {±id}. In 
this case, C^°(a; W) (resp. £'(a; W)) is the space of smooth functions (resp. distributions) 
/ on R so that conv(supp /) is a bounded interval of the form [-R, R] for some R > 0. The 
subspace C^°(a) w consists of the compactly supported even smooth functions on R; with x 
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equal to the sign character, the subspace C^°(a; x) consists of the compactly supported odd 
smooth functions on R. 



2. The support theorem 

In this section we proof the main result of this article. This is the following version of the 
the support theorem of Lions and Titchmarsch: 

Theorem 2.1 (The M^-Invariant Support Theorem). Let ^ X C a be open, convex and 
W -invariant. Suppose D is a W -invariant linear partial differential operator on a with real 
analytic coefficients and with principal symbol of the form (Q7J). Then 

conv (supp Du) = conv(suppii) 

for eachue S'(X;W). 

We shall prove below that, by regularization, Theorem 12.11 is equivalent to the following 
smooth version: 

Theorem 2.2. Let X and D be as in Theorem \2.1\ Then 

conv ( supp Df) = conv(supp /) 

for each f G C™(X;W). 

Equivalently, for each f G C%°(X; W) and for every compact convex W -invariant subset 
C C X we have 

supp Df C C -<=>- supp / C C . 

Proof of the equivalence of Theorems \2.1\ and \2.2\ It is clear that Theorem 12.11 implies 
Theorem 12.21 For the other direction, let u G £'{X; W). Let {ip £ : e > 0} be an approximate 
identity with supp^ e C B £ for all e > 0. Since X is a VT-invariant, we can, if necessary, 
replace ipe by r^r J2 w& w( w '"^e)- We can therefore suppose that ip £ is PF-invariant. It follows, 
in particular, that supple and hence conv (supple) are ^-invariant. Thus, by (JHJ), 

conv ( supp {u * ipe)) = conv ( supp u) + conv (supple) 

is a VT-invariant compact subset of a. Since X is open, we can choose Eo > so that 
u * %j) e G C^°(X; W) for < e < e . The functions u * ip £ converge to u in £'(a) as e — > 0, 
and D(u * ip £ ) = Du * ip £ . It is easy to check (without invoking deeper support theorems) 
that there exists 5(e) > 0, converging to as e — >• 0, so that 

supp u C supp (u * ip e ) + B S ( £ ) . (11) 

Replacing u with Du, we obtain 

supp (Dm * ipe) Q supp Du + B £ , 

SUpp Du C SUpp(Du * 1p £ ) + Bs( e ) ■ 

The same inclusions are preserved for the convex hulls. By Theorem 12.21 we have 

supp(w * ip £ ) = supp(D(w * ip £ )) . 
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Hence, 



conv(suppw) C conv(supp(w * ip e )) + Bs( e ) 

= COnv ( SUpp(Dll * Ipe)) + B S (e) 

C conv(supp.Du) + B e+S ( e ), 

which implies that conv(supp u) C conv(supp Du). Similarly, conv(supp Du) C conv(supp w). 
This proves Theorem 12.11 |-| 

Before going into the details of the proof of Theorem 12 .2\ let us briefly explain the main 
ideas involved. Since supp Df C supp /, it suffices to show that if C is a compact convex 
PF-invariant subset of X, then supp Df C C implies that supp / C C. This will be proved 
by contradiction. The main tool will be Holmgren's uniqueness theorem. 

Theorem 2.3 (Holmgren's uniqueness theorem). Let ^ H C 1" be open, and let if be a 

real valued function in C 1 (fi). Let D be a linear partial differential operator with analytic 
coefficients defined in Q. Let o~{D) denote the principal symbol of D. 
Suppose that x is a point in Q such that 

a(D)(x o ,d<p(x o ))^0. (12) 

Then there exists a neighborhood Q' CQ of xq with the following property: If the distribution 
u G T>'(Q) is annihilated by D, i.e., Du = 0, and u vanishes on the set {x G Q : (p(x) > 
ip(xo)}, then u must also vanish in Q'. 

Proof. This is Theorem 5.3.1 in |Hoer64j . |-| 

For non-elliptic differential operators the delicate matter is to choose points xq for which 
condition ()12j) is fulfilled. This is exactly the kind of difficulty one encounters in the proof 
of support theorems. Coming back to the streamline ideas of the proof of Theorem 12.21 let 
C be as above. Set S := supp/. Then S C X. To reach a contradiction, we assume that 
S % C . Let x E S \ C. As C is convex and compact, there exists a hyperplane strictly 
separating x and C, i.e., we can find A 66 o* such that 

maxAo(y) < A (x ). 

yec 

Without loss of generality we can also assume that 

^oO^o) — maxAo(x) . 

Otherwise we translate the hyperplane to the boundary of S in the direction opposite to C . 
In this way, the entire set S lies inside a closed half-space supported by the hyperplane 

Hq := {x G a : X (x) = A (x )} . 

Our plan is to apply Holmgren's uniqueness theorem to fl = X \ C, <p = Ao and u = f. Note 
that in this case dip = Ao is constant and non-zero. Observe also that Df = on Q and / = 
on the side of TCq not containing C (which is described by the equation <p(x) > ip(x )). If 
the principal symbol of D were not zero at (x , A ), then all assumptions would be satisfied, 
and we could conclude that / = in a neighborhood of x . This would yield the required 
contradiction because x G S = supp /. 
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a = X = M 2 , 

A = {±(1,0)} of type A u 

E+ = n = {(i,o)} 



Figure 1. 



Since A 7^ 0, the condition a(D)(x , A ) 7^ is equivalent to 

p(A )nK^o)r (a) ^0. (13) 

This might not be satisfied by the chosen pair (x , A ). It is even possible to have a situation 
where there is no choice of A for which the above procedure could guarantee that a(x ) 7^ 0. 
Figure 1 sketches an example in which this problem arises because of the "corner" at the 
boundary of S. Note that the set S in this example is also convex and invariant with respect 
to the group W = {id, r v }, where r v denotes the reflection with respect to the w-axis. 

In case p(A ) rLeA[ a (- r o)] n ^' ) = 0; the above procedure must be modified. The first step 
is to show that it suffices to consider the case of smooth functions / with the property that 
the convex hull C of S := supp/ is PF-invariant and has C 1 boundary. The point is that 
the xq, selected as above, will always belong to the boundary d{C') of C, and that Ho is a 
supporting hyperplane for C . For the modified procedure we need some preparations. 

For e > let B £ C a denote the closed ball centered at the origin and with radius e. Let 
B £ (C) := C + B £ be the (closed) e-neighborhood of C. 

Lemma 2.4. Let C be a proper convex subset of a = 1R™ with nonempty interior, and let 
e > 0. Then B £ (C) is an n- dimensional convex subset of a and its boundary dB £ (C) is a C 1 
(n — 1)- dimensional submanifold of a. 

Proof. This is Satz 17.6 in |NGL88j . D 

Lemma 2.5. Let D be a differential operator on 0. Suppose there exists a smooth compactly 
supported function f on a and a compact convex subset C C a with nonempty interior such 
that 

supp Df C C but supp / <f_ C . 

Then there exists a function f 6 C£°(a) and a compact convex subset C C a with nonempty 
interior such that 

supp Df C C and supp / <f_ C . 
Moreover, the boundary <9( conv(supp /)) of the convex hull of supp / is a C 1 -manifold. 

If f G C^°(X;W) , where X is a W -invariant open convex subset of a, and C C X is 
W -invariant. Then we can choose f G C£°(X; W) and C C X to be W -invariant. 
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Proof. Let {ip £ : £ > 0} be an approximate identity with snppip £ = B £ for all e. Then 

7* ip £ G C°° and 

supp (D(f * i) e )) = supp (Df * if> e ) C supp Df + B £ C B £ {C). 

Notice that (J3J) applied to compactly supported smooth functions implies that for all g G 
we have 

supp g C £> e (conv (supp g)) = conv(supp g) + £> e = conv (supp(g * VO)- (14) 

Hence, there exists eo > so that supp(/ * if) EQ ) % B eo (C). Otherwise (jUj) would imply that 
supp / C conv ( supp(/ * ipe)) Q B £ (C) for all e > 0, and hence supp / C C. As in (JHJ), we 
have 

conv (supp (/* ijj £0 )) = £ eo ( conv (supp/)). 
Therefore, by Lemma 12.41 conv (supp(/ * ip so )) has C^-boundary. We can thus select / = 

7*^o and C = B E0 (C). 

Finally, suppose that conv(supp/) and C are iy-invariants subset of X. Since X is 
open, then, by choosing a sufficiently small eo > 0, so are also conv(supp(/ * VO) = 
S £0 (conv(supp7)) and B £0 (C). n 

In the following we shall suppose that / and C are chosen as in Lemma 12.51 As before, 
we set S := supp / and C = conv (supp /). We also fix y G S \ C. We now proceed to the 
selection of the pair (xq, Xq). 

Lemma 2.6. Let G S\C be fixed, and let 

U := {A G a* : maxA(y) < X(y )}. 

y ec 

Then Uq is a nonempty open subset of a*. 

Proof. The separation properties of compact convex sets ensure that Uq 7^ 0. Observe that 

U = {A G a* : maxX(y - y ) < 0} 

= {A G a* : min \(y) > 0}. 
y&ya-C 

Hence Uq = /i _1 (]0, +00 [) where h(X) := mm y£yo _c X(y) . This proves the lemma because h 
is proper and convex, hence continuous (see |Rock70j . Theorems 5.3 and 10.1). rj 

Since Uq is open and nonempty we can choose Ao G Uq with the following properties: 

(1) P(A ) + 0, 

(2) (A , a) ^ for all a G A (i.e. X (x a ) ^ for all a G A). 
Since C = conv S, 

Lemma 2.7 (Choice of (xq,Xq)). Let Xq be as above. Then there exists Xq G S such that 
Ao(^o) = m&x x£ c' X (x). Furthermore x G dS R dC and x G X . Finally 

Xq(x ) = maxAo(x) > X (y ) > maxX (y). 

xeS y&C 

Proof. This follow as C is the convex hull of S. □ 
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When A ^ I, the lU-invariance of the situation plays a role because of the following 
lemma. 

Lemma 2.8. Suppose C is a W -invariant compact convex subset of a with nonempty interior 
and C 1 -boundary. Let a G A and x G dC such that a(x) = 0. Let T x (dC) denote 
the tangent space to dC at x (regarded as subspace of a) and let x a be as in J3J). Then 
x a eT x (dC>). 

Proof. The reflection r a of a across the hyperplane kera maps C onto itself and fixes x. 
Hence it maps (C")° (resp. dC) onto itself. The derived involution r a is therefore an 
automorphism of T x (dC f ). Hence r a (N) = ±iV, where iV is the outer normal vector to 
dC at x. Since r a N ^ —N by invariance of C under r a , we conclude that r a N = N, i.e. 
iV G ker a = . Thus x a G x^ x C N x = T x (dC). n 

Proposition 2.9. Let xq and Xq be as in Lemma \2. 7| Then (xq, Xq) satisfies (Q3J). 

Proof. The element Ao has been chosen so that p(Ao) ^ 0. This suffices to prove (fTTty 
when A = 0. If A ^ 0, then it remains to show that a(xo) ^ for all a G A. Since 
^o(^o) — m aXa; gC </ A (x), the hyperplane TLq := {x G o : X (x) = A (x )} is the supporting 
hyperplane for C := conv(supp/) at Xq. The set C is H^-invariant and its boundary dC is 
C 1 . Hence H = x + T xo dC. If a(x ) = for some a G A, then x a G T xo dC by Lemma |2~H1 
i.e. x + x Q G 7Y - Thus A (x ) + A (x a ) = Ao(x + x Q ) = X (x ), which implies X (x a ) = 0, 
against our choice of Ao- Thus a(xo) ^ for all a G A. |-| 

Proof of Theorem \2.Sl Arguing by contradiction, we assume that there exists / G C^°(X; W) 
and a H^-invariant subset C of X as in Lemma 12.51 We select (xo, Ao) as in Proposition 
12.91 Set Q = X \ C and ip = A . Hence Ho := {x G Q : A (x) = A (x )} is a supporting 
hyperplane for C := conv(supp /), and dip = Xq is constant and non-zero. Moreover, Df = 
on Q and / = on the side of 7io not containing C (which is described by the equation 
Xq(x) > Ao(x )). Proposition 12.91 ensures that a(D)(xo, Ao) ^ 0. Holmgren's Uniqueness 
Theorem then implies that / = in a neighborhood of xq. This gives the the required 
contradiction because xq G supp /. □ 

Before concluding this section we prove some immediate consequences of Theorem 12.11 
Recall that the transpose of the partial differential operator D given by (jlj) is 

D= (-l) m a/(x)<9'. (15) 

\I\<m 

Hence D l belongs to the class of invariant differential operators considered in this papers if 
so does D. In fact, the principal symbols of D and D l are linked by the relation 

«t(D*)(x, A) = (—l) rn a(D)(x, A) , (x, A) G a x a*. 

Corollary 2.10. Let D be a W -invariant differential operators D on a with real analytic 
coefficients and with principal symbol of the form il(J\) . Then the following properties are 
true. 

(a) D is injective on C^°(X; W). 

(b) For all f G C C °°(X; W) we have 

conv ( supp D t f\ = conv(supp /) . 
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3. Applications to some special differential operators 



In this section we present some examples linked to the theory of special functions associated 
with root systems. In these examples the group W is a parabolic subgroup of a fixed Coxeter 
group W acting on a ~ R ra , and the differential operator D is in fact invariant under the larger 
finite Coxeter group W. The general situation corresponds to a (not necessarily reduced) 
root systems E. A root system is a finite set S C a* \ {0} satisfying condition (R2) of Section 
[0 In this section we will also assume that E satisfies the following additional conditions: 

(R0) E spans a*; 



(R2) E is crystallographic, that is 



2f^— '- g z 

{a, a) 



for all a, [3 G E . 

Crystallographic root systems arise naturally in several places in algebra and analysis. In 
particular, they are relevant in the theory of real Lie algebras, Lie groups, and symmetric 
spaces. 

If E is a root system, then A := {a G E : 2a ^ £} is a reduced root system according 
to the definition of Section ^ The finite Coxeter group associated to E is the finite Coxeter 
group W associated to A. It is also called the Weyl group of E. A multiplicity function is a 
IV-invariant function m : E — > C. For a G E, we adopt the common notation m a to denote 
m(a). 

The set E + of positive roots and the set II of positive simple roots can be defined as in 
the case of reduced root systems. Because of (R0), II is a basis of a*. Fix a II set of simple 
roots in E + . For each subset of IT we define (0) as the set of elements of E which are 
linear combinations of elements from 0, i.e., (0) := Z0 H E. It is itself a root system, but in 
general it does not satisfy (R0). We denote the corresponding finite Coxeter group by W e . 
Note that W e C W is generated by the reflections r a with a G 0. For instance, W n = W 
and W 9 = id. We also set (0) + := (0) H E + for the set of positive roots inside (0). The 
Coxeter group Wg_jwill play the role of the group W of the previous sections. Recall that 
a subgroup W of W is called a parabolic subgroup if it is of the form W e for some C IT. 
The parabolic subgroups can also be characterized as those subgroups of W that stabilize a 
subspace of a. Thus W C W is a parabolic subgroup if and only if there exists a subspace 
b C a such that 

W = {w G W : w(b) = b}. 

A chamber in a a connected component of a \ U ae s7^ a (sec [Bou02j, Ch. V, §3). ^From 
now on we fix the chamber a + := {x G o : a(x) > for all a G 11} . It is an open simplicial 
cone with vertex (loc. cit, Ch. V, §3.9, Proposition 7 (iii)), and its closure a + := {x G a : 
a(x) > for all a G LT} is a fundamental domain for the action of W on a (loc. cit., Ch. V, 
§3.3, Theorem 2). 

We define 

a e := W e (^)°, (16) 

where denotes the interior. For instance, a = a + and a n = a. In the following examples, 
the sets a e will play the role of the set X appearing in Theorem 12.11 
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The set a e = W (a + ) is the smallest cone in a which is closed, We-invariant and contains 
a + . It is a union of closed chambers. The open polyhedral cone in a* 

C e : = R+Xa 

aGS+\{8) + 

has the dual cone 

C* : = {x E a : (x, y) > for all y E C @ } 

= {x E a : a(x) > for all a E S + \ (0)+}. 

C* is a closed convex cone. It is the intersection of the closed hyperplanes defined by roots, 
and hence a union of closed chambers in a. 

Lemma 3.1. We have 

Consequently, is a closed W e -invariant convex cone in a and also its interior a e is convex. 
Moreover, 

a e = {x E o : a(x) > for all a E S + \ (0)+}. (17) 

Proof. This was proven in |Pa04j . Lemma 3.4, when W is a Weyl group. The same proof 
applies also to the more general case of finite Coxeter groups. □ 

Specializing the notation of Section [T] to this context, we consider the space C^°(a e ; W e ) 
of smooth functions / : a — » C with the property that conv(supp/) is a We-invariant 
compact subset of a e . Its subspace of -invariant functions on a e with compact support is 
C™(a e ) We . Furthermore, £'(a e ; W e ) is the space consisting of distributions u on a so that 
conv(suppw) is a We-invariant compact subset of a e - 

3.1. Hypergeometric differential operators. As before let o ~ 1™ be a finite dimen- 
sional Euclidean space and let S be a (non-necessarily reduced) root system in a*. Further, 
let W the be the corresponding Weyl group and let m be a multiplicity function on S. Heck- 
man and Opdam associated with such a triple (a, S, m) a commutative family of W-invariant 
differential operators on a, the hypergeometric differential operators, having meromorphic co- 
efficients of a specific type. More precisely, their coefficients are meromorphic functions on 
the complexification ac of a and their singularities are cancelled by multiplication by a 
suitable power of the Weyl denominator 

5(x) := Yl sinha(x). (18) 

We refer to [He c97j . |UpdOO| and |HS94j for more information on Heckman-Opdam's theory 
of hypergeometric differential operators. 

For special values of m, the triple (a, E, m) arises from a Riemannian symmetric space 
G/K of the noncompact type, i.e., G is a noncompact connected semisimple Lie group with 
finite center and K C G is a maximal compact subgroup, see Section In this case, the 
hypergeometric differential operators coincide with the radial parts (with respect to the in- 
action) on a + of the G-invariant differential operators on G/K. Here we identify a + with 
its diffeomorphic image in G/K under the exponential map (usually denoted A + in the 
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literature on analysis on symmetric spaces). For instance, the hypergeometric differential 
operator 

L := L a + m a cotha d(y a ) (19) 

coincides with the radial part of the Laplace-Beltrami operator of G/K. Notice that L is 
singular on the hyperplanes 7i a , a G E. Furthermore, L is the multivariable analog of the 
Jacobi differential operator of Example 11.31 In ([19)1. d(y) denotes the directional derivative 
in the direction of y G o, and L a is the Laplace operator on the Euclidean vector space a, 



that is. 
Set 



L a = Y^j=i d(xj) 2 where {xj}™ =1 is an orthonormal basis of a. 



ir e {x 



S (x) :-- 



5%{x) :-- 



n 

ae<e>+ 

] [ sinh a(x) , 
ae(e>+ 

| [ sinh a (x) 

aGE+\(G)+ 



with the usual convention that empty products are equal to 1. We write 7r(x) instead of 
7Tn{x)- Finally, define D := 5 2 ■ L. The principal symbol of D is 

a(D)(x, A) = 5(x) 2 ■ a(L a )(x,X) 

where 

P(X,x) := [5 c e (x)] 2 



(A,A)vr e (x) 2 P(A,x) 



m\ • (2o) 

Because of Lemma l3~T| each a G S + \ (0) + is positive on a e . Therefore P(X,x) is a homo- 
geneous de-elliptic polynomial on o* x a (of degree in A G a*). More generally, for each 
hypergeometric differential operator Dq there is k G N so that the linear partial differential 
operator D := 5(x) 2k D is a W-invariant differential operator with real analytic coefficients 
and with principal symbol of the form (|1(J|) where P(A, x) is a homogeneous a e -elliptic poly- 
nomial. In this case, we say that D = 5(x) k Do is a regularization of Dq. The nature 
of the principal symbol of D can in fact be deduced by the explicit representation of the 
hypergeometric differential operators in terms of Cherednik operators (see e.g. |Hec97j ). Ac- 
cording to this representation, each hypergeometric differential operator can be constructed 
as the "differential part" of the differential-reflection operator associated with a ^-invariant 
polynomial in the symmetric algebra over ac by means of Cherednik operators. If Dq is 
associated with the VF-invariant polynomial p, then the principal part of the regularized 
operator D = 5(x) 2h ■ Dq is 

2k\K^ 2k 



a(D)(x,X)=p h (X)7r(x) 



X 



7r(x). 

2k P(X,x 



= Ph(X)7T, 

where Ph(X) is the highest homogeneous part of p(A) and 

-5, 



P{\x) := [5 c e (x)) 



2k 



[X 



L7To (X J 



2A: 
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In this setting, Theorems 12.11 and 12.21 yield the following result. 



Theorem 3.2. Let the notation be as above. Let C IT. For a hypergeometric differential 
operator Dq let D = 5(x) 2k ■ Dq be a regularization of Dq. Then for every u G S'(a @ ; W e ) we 
have 

conv(suppDw) = conv(suppw). 
Equivalently, for every f G C£°(a e ; W e ), we have 

conv(supp Df) = conv(supp /). 

In the case of W e -invariant functions, Corollary 13.21 can be stated directly for the hyper- 
geometric operators. 

Theorem 3.3. Let the notation be as above. Let Dq be a hypergeometric differential operator 
and f G C™(Oe) w& . Then D Q f G C™(a e ) w ° for all f G C™(a @ ) w& and 

conv(supp D f) = conv(supp /) . 

Proof. As 

a = (J w(a + ) = (J iw(ao) 

wgW weW/We 

and supp(/) C Oe is compact and We-invariant, it follows that there exists a unique / G 
C c °°(a) # so that f\ a@ = f. Let D = 5 2k ■ D be a regularization of D . Then 

( J D/)| ae = J D(/| ae ) = J D/. (21) 

Suppose Dq is the differential part of the Cherednik operator T (see e.g. |Hec97j ). Then 
Dof = Tof by VF-invariance. Cherednik operators map H^-invariant smooth functions into 
1^-invariant smooth functions. It follows that D f is smooth and ^-invariant. ^From (|21j) 
we therefore deduce that 

{5 2k -Dj)\ ae 
u of p 

extends to be smooth and jy-invariant on a e . If g is a continuous function, then 5 2k ■ g and 
g have the same support. The Theorem therefore follows. □ 

3.2. Bessel differential operators. Let G be a connected noncompact semisimple Lie 
group with fintie center and K C G a maximal compact subgroup. Then K = G e for 
some Cartan involution 9. The Lie algebra q decomposes into eigenspaces of the derived 
homomorphism 9 : Q — » g: 

= «©P 

where t is the +1 eigenspace and p is the —1 eigenspace. Consider p as a abelian Lie group. 
The group K acts linearly on p by k-X = Ad(k)X. We can therefore consider the semi-direct 
product 

G = p x Ad K. 

Let (o, E, m) be as in Example 13.11 The Bessel differential operators associated with 
(o, E, m) are the "rational" analogs of the hypergeometric differential operators. They are 

16 



jy-invariant and the singularities of their meromorphic coefficients are cancelled by multi- 
plication be a power of the polynomial 

ir(x) := Y[ a(x) . (22) 

When the triple (a, £,m) arises from the structure of Riemannian symmetric space G/K of 
the noncompact type, the Bessel differential operators coincide with the radial parts on a + 
of the Go-invariant differential operators on the corresponding Riemannian symmetric space 
Gq/K of Euclidean type. For instance, the Bessel differential operator 

L := L a + m a -d(y a ) (23) 
^— ' a 

coincides with the radial part of the Laplace-Beltrami operator on Gq/K. The operator L 
is the multivariable analog of the Bessel differential operator of Example 11.31 The Bessel 
differential operators can be constructed as "differential parts" of differential-reflection op- 
erators associated with W-invariant polynomials functions p on a£ by means of the Dunkl 
operators |d.T04j . Suppose D is the Bessel differential operator associated with p. Let k G N 
be chosen so that D := -n-(x) 2k D has real analytic coefficients. Then the principal symbol 
of D is 

a(D)(x, A) = Ph (X)n(x) 2k = p h (X)n e (x) 2k P(X, x) , 

where Ph(X) is the highest homogeneous part of p(X) and P(X,x) = riaeE+\(e)+ a ( x ) 2k is a 
homogeneous o e -elliptic polynomial of degree in A. As for the hypergeometric differential 
operators, we obtain the following corollary of Theorems 12.11 and 12.21 

Theorem 3.4. Let the notation be as above. Let O be a fixed set of positive simple roots in 
S. For a Bessel differential operator Dq let D = 7r(x) 2k ■ Dq be a regularization of Dq. Then, 
for u G £'(a e ; W B ) we have 

conv(supp Z>u) = conv(suppw). 
Equivalently, for every f G C^°(o e ; W e ), we have 

conv(supp .D/) = conv(supp /). 

IffE C-(a e ) We ; then D f G C~(a e ) We and 

conv(supp -D /) = conv(supp/) . 

3.3. Shift operators. For the study of hypergeometric differential operators corresponding 
to different multiplicity functions (see Example 13 Opdam introduced certain TV-invariant 
differential operators, called shift operators. For simplicity, we treat here only the case of 
a reduced root system S and we refer the reader to [HS94 , Part I, Ch. 3 for the general 
case. As in the case of hypergeometric differential operators, Opdam's shift operators have 
meromorphic coefficients with singularities along the root hyperplanes, and their singularities 
are cancelled by multiplication by powers of A. A shift is a multiplicity function / which 
is even (i.e. such that l a G 2Z for all a G E). Opdam's shift operators exists for any shift 
/. They can be considered as generalizations of the hypergeometric differential operators, 
which form the space S(0) of shift operators of shift 0. For an arbitrary nonzero shift I, the 
space iS(/) of shift operators of shift / is a free rank-one (right) (S(O)-module generated by a 
certain shift operator G(l). See loc. cit., Theorem 3.3.7. Suppose furthermore that either 
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l a > for all a or l a < for all a. Then G(7) is obtained by composition of fundamental 
shift operators Gg t ±, shifting of ±2 the multiplicities along each Weyl group orbit (loc. cit., 
p. 42). The principal symbol of each fundamental shift operator is 

<j{G S)± )(x,X) = c-A s {x) ±1 J] \(x a ), 

where S + = S + nS and A$(x) = Y[ a ^s+ sinha(x). See loc. cit., Remark 3.3.8. The analysis 
done in Example 13.11 for the hypergeometric differential operators, in particular Corollary 
13.21 easily extends also to the fundamental shift operators, and hence to all shift operators. 

A similar argument can be also applied to the shift operators associated with the Bessel 
differential operators. For more information on the latter we refer to [Hcc97j, where they 
are studied in the general case in which W is an arbitrary finite Coxeter group. 



3.4. Hamiltonian systems. A wide class of integrable Hamiltonian systems associated 
with root systems were introduced by Olshanetsky and Perelemov in |UPe76j . Let (a, £,<?) 
be a triple consisting of a finite dimensional Euclidean space a, a root system E in a*, and 
a real- valued multiplicity function g on E. These integrable systems are described by a 
Hamiltonian of the form 

H = ~-{p,p) + U(q), p,qea, 

with potential energy 

U(q) := 

where the function v has five possible forms. Here we consider the cases I, II and V as listed 
in [MO], (3.1.14) and (3.8.3): 

(I) v(0 = C\ 

(II) i;(0 = sinh- 2 e, 

(v) V (o = c 2 +^e. 

The Calogero Hamiltonian of Example 11.41 is a special instance of case I. The associated 
Schrodinger operators are the linear partial differential operators 

S = --L a +U(x), xea. 

They are invariant with respect to the Weyl group W of E. The operator S can be regularized 
by multiplication by the polynomial tt(x) of ()22|) in the cases I and V, and by multiplication 
by 5 as in (|18|). The possibility of applying Theorems 12.11 and 12.21 to these regularized 
differential operators can be proven as in Examples 13.11 and 13.21 In fact, there is a close 
relation between the operator S of case II, resp. of case I, and the operator L of (JTHJ), resp. 
to the operator L of (J23J). See e.g. |HS94] . Theorem 2.1.1, and j()Pe76j . 

For an overview on the role of the Hamiltonian systems treated in this example in different 
areas of theoretical physics and mathematics, we refer the reader to [DVOOj . 
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4. Applications to symmetric spaces 



In this section we apply Theorem 12.21 to differential operators on symmetric spaces. In par- 
ticular we give a new proof of the D-convexity of the Riemannian symmetric space G/K, 
c.f. [He 73 , vdBS93 . Our proof uses Theorem 12.21 applied to the radial part of invariant dif- 
ferential operators and is, as far as we know, new. Our notation is the same as in Subsection 

EH 

Let us start by recalling the notation from Subsection 13.21 Here G is a connected non- 
compact semisimple Lie group with finite center. Furthermore 9 : G — > G is a Car- 
tan involution and K = G e the corresponding maximal compact subgroup. Denote by 
k : G — > G/K, the natural projection g \— > gK. We denote also by 6 the derived involution 
on g. Then g = t © p, where t is the +l-eigenspace of 9, and p the — 1-eigenspace. Then 
6 is the Lie algebra of K. Fix a Cartan subspace a, that is a maximal abelian subspace 
of p. The Killing form on g defines an Euclidean inner product on a. For a G a* set 
g a = {y G g a : [x,y] = oi{x)y for all x G a}. Then the set S consisting of all a G a* \ {0} 
for which g a ^ {0} is a (generally non-reduced) root system as defined in Section El It is 
called the (restricted) root system of (g, a). The multiplicity m a of a G £ is defined as the 
dimension of g a . The map m given by m(a) := m a is a multiplicity function on S. This 
construction associates with the Riemannian symmetric space G/K a triple (a, E, to). As 
before we denote by W the corresponding Weyl group. 

Let S + be a choice of positive roots and let a + := {x G a : a(x) > for all a G S + } be 
the corresponding positive chamber. Denote by exp : g — > G the exponential map. Then 
A := exp a is an abelian subgroup of G diffeomorphic to o. We set A + := expo + . The 
map K x A x K 3 (ki, a, k 2 ) ^ kiak 2 G G is surjective and the A-component is unique up 
to conjugation by an element of W. Hence every i^-bi-invariant subset of G is of the form 
K(expB)K where B is a IV-invariant subset of a. Moreover, B is compact if and only if 
k(B) C G/K is. Let C^°(G/K) denote the space of compactly supported smooth functions 
on G/K. Using the map C™(G/K) -> C™(G/K) K , /h4/o ft ,we will often identify smooth 
functions on G/K with the corresponding right ^-invariant function on G. 

The decomposition G = KAK yields the following lemma. 

Lemma 4.1. Let f G C%°(G/K) and suppose that supp / is K-bi-invariant. Then supp / = 

ii"(supp }\a)K where /[a denotes the restriction of f to A. Moreover, supp /[a is a W- 
invariant compact subset of A. 

Denote by B>(G/K) the (commutative) algebra of G-invariant differential operators on 
G/K. We identify A + can be identified with the submanifold k(A + ) C G/K. Then, for 
every D G 3(G/K), there is a unique W- invariant differential operator, u(D) on A + , called 
the radial part of D, so that for all / G C°°(G/K) one has 

(Df)\ A+ =uj(D)(f\ A+ ) (24) 

See |Hel84| . p. 259. 

Define a differential operator on a, also denoted by uj(D), by 

w(D)g := u(D)(g o exp" 1 ) o exp , g G C°°{<x) . 

In this way, we can consider u(D) as a (singular) H^-invariant differential operator on the 
Euclidean space a. To simplify our notation, we shall adopt the identification A = a using 
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the exponential map. Then exp := id. We then write /| instead of /\a, and the above 
mentioned decomposition of a if-bi-invariant subset of G will be written as KBK instead of 
if (exp B)K. With these identifications, the operator u(D) turns out to be a hypergeometric 
differential operator associated with the triple (a, E, m) as considered in Subsection 13.11 
When / G C^°(G/K) is if -invariant, we can moreover extend (}2"4"j) by W^-invariance to 
obtain 

(Df)\ a = co(D)(f\J (25) 

Lemma 4.2. Let G/K be a Riemannian symmetric space of the noncompact type. Let 
D G TD(G/K) and let B be a compact, convex and W -invariant subset of a. Set Xb '■= KBK . 
Then for all f G C™{G/K), we have 

supp / C Xb if and only if supp(-D/) C Xb- 

Proof. We need to prove that supp(.D/) C X B implies supp / C X B . The first step is, as in 
vdBS93j, to reduce to the case in which the support of / is left- if -invariant. The function 
/ can in fact be expanded as a sum of if -finite functions. Since Xb is left- if -invariant, 
we will obtain supp / C Xb if the support of each if -finite summand is contained in Xb- 
We can therefore assume that /, and hence Df, are if -finite. As the support of a if- 
finite function is left if -invariant, we obtain from Lemma 14. II that supp / = if (supp f\a)K 
and supp(DJ) = if ( supp(D/)| )if where supp/| a and supp(D/)| a are jy-invariant and 
compact. Since supp(£>/) C X B , we then conclude that supp(_D/)| C B. Since u(D) is a 
hypergeometric differential operator, Corollary 13 . 31 with = II yields that conv(supp /|„) = 
conv(a;(D) supp/| a ) = conv(supp(L>/)| ) C B. This proves the required inclusion. ^ 



Theorem 4.3. Let G/K and D G D(G/if) be as in Lemma \4.^\ Then G / K is D -convex, 
that is for every compact subset S of G/K there is a compact set S' so that for every 
f G C%°(G/K) the inclusion supp(D*/) C S implies supp/ C 5". Here D l denotes the 
formal transpose of D. 

Proof. Choose Si C G so that S\/K — S The set KS\H is if-bi-invariant, hence if 5*1 if = 
Xb '■= KBK for some ^-invariant compact subset B of A = a. Since D t G D(G/if) we 
can apply Lemma 14.11 to it. So supp(/) C X convB - We can then select 5" := X convB /K. ^ 

We conclude this section by a short discussion of the application of the theorem of our 
supp ort th eorem in harmonic analysis corresponding the the 0-hypergeometric transform, 



c.f. OP04 . Because of the technical nature of this application, a detailed exposition would 



require a certain amount of notation and of background information. Inste ad, we prefer here 



just to outline the main ideas involved. We refer the interested reader to |OP04j for further 
information. 

As already remarked in the introduction, the support theorem plays a role in the study of 
the Paley- Wiener space. This is the set of images under a suitable generalization T of the 
Fourier transform, of the compactly supported smooth functions. The Paley- Wiener space 
generally consists of entire or meromorphic functions with exponential growth and possibly 
satisfying additional symmetry conditions. The size the support of the original function is 
linked to the exponential growth of its Fourier transform. The thrust of Paley- Wiener type 
theorems is usually to prove that, if a function g in the Paley- Wiener space has a given ex- 
ponential growth, then the support of the associated "wave packet" Tg (obtained by formal 
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application to g of the inverse Fourier transform X of J-) is compact and has the correct size. 
In the classical situation of the Fourier transform on M. n , this is proven by a suitable "shift" 
of contour of integration by means of Cauchy's theorem. The shift is allowed because Xg 
is given by integration of an entire function of exponential type and rapidly decreasing. A 
suitable generalization of this argument was applied also to the spherical Fourier transform 
on Riemannian symmetric spaces. See |Hel84j . Ch. IV, §7.2, or [GV88 , §6.6. For pseudo- 
Riemannian symmetric spaces the situation is more complicated. Here the wave packages are 
integrals of meromorphic functions and the required shift of integration would generally re- 
quire addition of certain "residues" . For some kinds of Fourier transforms, like the spherical 
Fourier -Laplace transform on noncompactly causal symmetric spaces with even multiplicities 



OP04j , the singularities of the integrand are cancelled by multiplication by a certain polyno- 
mial function p. Hence, for every function g in the Paley- Wiener space the wave-packet X(pg) 
is given by integration of an entire function. Furthermore, there is an invariant differential 
operator D so that DiXg) = X(pg). The possibility of shifting the contour of integration 
in X{pg) allows us then to determine the size of the support of D{Xg). The generalization 
of the support theorem of Lions and Titchmarsch presented in this papers allows us finally 
to recover the size of the su pport o f Xg. A concrete application of this procedure using 
Theorem 12 . 21 can be found in [OP04 in the context of the B-hypergeometric transform. The 
latter transform, stated in a setting of transforms associated with root systems, is a common 
generalization of Opdam's hypergeometrical transform (hence of Harish-Chandra's spherical 
transform on Riemannian symmetric spaces of noncom pact ty pe) and of the Fourier-Laplace 



transform on noncompactly causal symmetric spaces H096j. A similar application would 



also be needed for determining the Paley- Wiener space for the transform associated with the 
6-Bessel functions of |BSO04| . 
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